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ABSTRACT 

The question of what variables affect success in 
debate has long been an area of interest and concern in the forensic 
community. For many years, it was thought that traditional 
performance variables-delivery, reasoning, organization, analysis, 
refutation and use of evidence— were the key factors influencing 
evaluations of debaters. Some authorities have suggested that 
broader, more general standards dictate evaluation. These factors 
include geographical variables and gender. Some authorities have 
abandoned attempts to determine the weight of specific variables and 
embraced a holistic approach to debate speaker evaluation. Although 
the trend is toward holistic evaluation, additional research is 
warranted; such a movement may not be justified. It might be an 
interesting personal experiment for judges confronted with a ballot 
that offers boxes" to attempt to mark them and to see if that 
evaluation IS consistent with their holistic evaluation. (Seventy 
notes are included.) (Author/RS) 
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Abstract - "Holistic Speaker Evaluation-A Review and Discussion" 



The question of what variables effect success in debate has long been an area of interest 
and concern in the forensic community. For many years, it was thought that traditional 
performance variables-^delivery, reasoning, organization, analysis, refutation and use of 
evidence-were the key factors influencing evaluations of debaters. Some authorities have 
suggested that broader, more general standards dictate evaluation. These factors include 
geographical variables and gender. Some cities have abandoned attempts to determine the 
weight of specific variables and embraced a holistic approach to debate speaker evaluation. This 
paper reviews the literature surrounding the use of traditional indicators of speaker success. 
Other variables that may influence evaluation and success are also examined, A detailed 
discussion of the need for further research in this area is offered. There is, finally, a discussion 
of non-research interests as well. 
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To paraphrase Quintilian, an outstanding intercollegiate debater is simply a "good person 
debating well." Unfortunately, little consensus has emerged as to what combination of elements 
constitute "debating well." There has been a longstanding controversy in the debate community 
surrounding this question. Forensics scholars have long been uncertain as to whether specific, 
observable elements account for good debating and successful debating, or whether more 
abstract, non-performance factors play a more central role. 

Much of the intercollegiate debate community has, at least implicitly, endorsed a standard 
evaluation form which suggests that six factors are of the greatest importance in debate 
performance: delivery, reasoning, organization, analysis, refutation, and use of evidence. 
These six factors are included on the American Forensic Association's Form C debate ballots 
to facilitate uniform evaluation of debate speakers. Many other debate ballots utilize similar 
variables. 

Pi evious literature has endorsed the validity of utilizing the six factors on the Form C 
ballot for evaluation. For instance, Professor Burgeon found that a "correlation analysis" 
computed among the six predictor variables and the criterion variable, "revealed that actually 
all of the six predictor variables by themselves were significantly related to percentage of wins. 
She went on to note that "while organization and refutation emerged as being slightly more 
important, all six factors were relatively equal in their impact."^ 

Other scholars have also recognized the relevance of the six Form C factors. "The 
Williams, Clark, and Wood findings suggest that the traditional criteria have a major impact." 
although they do go on to note that "they are not independent."^ Professor Giffin of Kansas 
conducted a study which found elements very similar to these traditional six, as constituting the 
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majority of evaluative criteria employed by debate judges. Giffin explained the results in this 
fashion: 

... .the criteria employed in each debate by each judge were tabulated; of the total 
consideration given to all different criteria it was found that the judges gave to 
each of the criteria included in our hypothesis the following weight or 
consideration: 



1. ability to speak well (delivery) 14.65% 

2. selection of logically defensible 
arguments (case) 19.10 

3. support of arguments with 

information (evidence) 17.18 

4. perception of irrelevant or irrational 
arguments (refutation) 17.00 

5. phrasing of concepts clearly and 
concisely (language) 5.29 

6. ability to analyze the topic area 
(analysis) 14.78 

7. ability to organize ideas into a 

structured whole (organization) 8.88 

96.88%^ 



Whether each of the six traditional factors independently weigh upon a judge's evaluation 
and decision is still open to question. For example, Professor Wise has suggested that the "two 
most difficult skills in academic debate, as measured by mean scores, are 'analysis' and (the use 
of) 'evidence.'"^ . The question of whether these two factors, or any of the other four, are 
actually more important, or whether they function synergistically would seem to warrant 
examination of each factor individually, 

Gerald Sanders has operationally defined reasoning "as the process by which we infer a 
conclusion from premises."^ Although Sanders does not attempt to quantify the relative weight 
that reasoning plays in a debate judge's evaluation, he does note that one should "emphasize the 
importance of reasoning in argumentation and the part that it plays in a judge's decision."*^ 
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Other authorities have suggested that reasoning is at least as important as a debater's use 

of evidence. Professor Cathcart has noted: 

....the speaker who skillfully incorporates into his own thinking the evidence 
gathered, and then weaves it smoothly into his speech, will be just as effective 
as, if not more so than, the speaker who stops to cite sources for all of his 
evidence, or the one who documents and qualifies each source.^ 

Again, reasoning is identified as important but the relative weight of such importance is stili 

unclear. 

One could surmise that reasoning would obviously be important as a debate skill, but the 
difficulty in attempting to independently measure its importance is equally obvious. The 
pervasive nature of reasoning in relation to debate may make it difficult to separate it from other 
factors. 

The great majority of contemporary forensic literature seems to place little value on the 
independent worth of delivery. Indeed, the conclusion reached by Vasilius and DeStephen seems 
quite true: "In debate the attitude toward delivery is ambivalent."^ Indeed, they went on to note 
that the "overall lack of significance suggests that a variety of factors contribute to debate 
success of which delivery, at least in quantitative terms, may be of little importance. "^^ Sanders 
has concurred by noting: "The judge who uses argumentation and logic as his sole criteria for 
determining the winner of an academic debate sees debate as an intellectual contest with speech 
being only an incidental element. "^^ 

There is actually a solid body of quantitative research which confirms the limited 
independent value that most debate judges and scholars assign to delivery. An analysis of 
judging philosophy statements found that: 
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Only a few critics indicated they 'generally give low points to spread debaters/ 
So long as debaters met basic requirements for intelligibility, most participants 
tolerated this form of discourse, 'believing the ultimate value of competitive 
debate to be analysis and not oratory. '^^ 

Similarly, delivery or "speaking ability" has been ranked extremely low in terms of its 

importance as an educational by-product of debate. Professor Pearce noted that: "A recent 

survey of attitudes toward forensics in the U.S. found that members of the American Forensic 

Association themselves ranked the development of speaking ability last in a list of educational 

objectives."'^ 

Delivery appears to be one factor of evaluation that clearly weighs less heavily than 
others. The consensus seems to be that it is not sufficient alone to determine the outcome or 
total performance evaluation of a debate. 

There is very little debate-specific literature in relation to the importance of organization. 

There is general literature concerning organization and speech communication. For example, 

Elaine Winkelman Butcher has observed: 

Results of some previous experimental studies indicated that speech organization 
did not contribute to message comprehension. Other studies claimed that 
credibility was not impaired by disorganization and that disorganization did not 
affect attitude. On the other hand, the majority of the literature as well as speech 
textbooks acknowledge the importance of speech organization.^'* 

However, Butcher has also noted that disorganization is not inherently negative or 
counterproductive. She noted: 

Results confirmed the importance of message organization on comprehension, but 
not on knowledge in some cases. Further, disorganization is detrimental to 
credibility only on those factors of qualification and safety, but not on warmth. 
Finally, this study showed no effect of message disorganization on attitude toward 
the topic. 
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The controversy over the importance of organization in relation to speech generally would 
seem to be relevant to debate as well. If judges are more concerned simply with the outcome 
of arguments, organization may not be key. However, good organization may very well effect 
the outcome of a given argument. Hence, the value of organizational ability as an independent 
factor in debate evaluation would appear to be open to question. 

••Analysis is," according to Sanders, "the arriving at an understanding of the proposition 
and the discovering of the issues inherent therein."'^ Newman has suggested that deliberative 
speakers, one would assume this could include the debater, "find that one of their most 
important tasks is analysis, or breaking a proposition down into its component parts."*'' 
Professor Rieke has applied the concept more specifically to debaters by noting that "analysis 
involves essentially two processes: discovering what basic questions must be asked in 
considering the resolution; and discovering what basic lines of reasoning are appropriate in 
setting about to answer the questions."*^ 

Analysis is another factor, like reasoning, that seems to be generally important, but very 
difficult to isolate and measure against other factors. Indeed, Professor Rieke's comment above 
clearly draws an interrelationship between analysis and reasoning, further complicating the 
situation. 

Evidence and evidence usage appear to be factors that have stimulated a good deal of 
debate-related literature. "Evidence is," notes Sanders, "an indispensable element in good 
debating and the argumentation and logic judge treats it as such."^^ In fact, a concern for 
evidence use is central to the selection of a debate resolution. Sanders, writing again, has noted: 
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"One of the criteria used for choosing an intercollegiate topic is that adequate evidence should 
be available on both sides of the proposition. 

According to William Dresser, "contemporary theorists generally agree that the use of 
carefully selected and tested evidence is important to the advocate.... "^^ There are many who 
feel this is particularly true for the debate advocate. "Championship level debaters," according 
to Benson, "not only use the greatest amount of evidence but also use a greater portion of their 
evidence to clash with their opponents by denying arguments or establishing counter 
contentions."^ Benson has quantified such usage levels: "The championship debaters, 
/operationally defined as those qualifying for elimination rounds at major tournaments/, use 
about 25% more evidence than the varsity level debaters, /operationally defined as those with 
one year or more experience/, and nearly 60% more evidence than novices."^ 

Although "championship" level debaters tend to use more evidence and evidence usage 
is generally recognized as important, there is no firm consensus on its value or effect. 
"McCroskey's findings," for instance, "that evidence is the least valuable factor for immediate 
attitude change" obviously casts doubt upon the inherent value of evidence usage.^^ "In debate 
situations," according to Vasilius and DeStephen, "where the critic must render an immediate 
decision, the quantity of evidence may be unimportant or at least not as important as other 
factors."" 

Many feel that evidence is interrelated to other factors and debating skills. Some authors 
have suggested "that evidence is used to support arguments and cannot be considered separate 
from the arguments. "^^ Professor Dresser has also suggested that evidence tends to work with, 
or aid other factors. He has reported that: 
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This study tends to support the position of those contemporary theorists who hold 
that the importance of carefully tested evidence in speech making lies not in its 
contribution to persuasiveness but in its usefulness in helping the speaker to 
explore his subject intelligently.^ 

The bottomline of contemporary forensic research seems to be that the value of evidence 

usage is simply uncertain. Kathy Kellerman summarized the situation rather succinctly: 

In contrast to the teachings of most introductory communication courses, 
theoretical consensus and empirical validation of the usefulness of evidence to a 
speaker have yet to be established. Indeed, the plethora of empirical research on 
evidence has produced such inconsistent results that no coherent theoretical 
perspective on the usefulness of evidence in argument can be extracted. 

Professor Sanders has defined the last of the six traditional standards in this way: 

"Refutation is considered to be the attempted destruction of the opponents' argumentation."^^ 

Sanders feels that refutation is one of the key elements that a judge considers in his evaluation 

of a debater. He has noted: 

In this area of the debate, the judge is watchful for a debater's exposure of 
weaknesses in the opposing case. Such weaknesses could be questionable analysis 
and interpretation, flaws in evidence, fallacies in structure and argument, and 
inconsistencies and contradictions in argument. 

There are others who have suggested that refutation is the single most important element for 

evaluation, "If any single measure could be applied to determine the potency of a debater,'* 

writes Professor Faules, "that measure would examine refutation skill. "^^ 

The results of actual debates seem to validate the relative importance of refutation. 

Faules noted that "winning debaters were scored superior more frequently for refutation than any 

other item. Such evidence indicates that refutation skill may be a predictor for debate 

effectiveness. "^2 Keeling also found that "the greatest difference in the scores of winning and 
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losing debaters occurred in the area of refutation. In addition, winning debaters were scored 
superior more frequently for refutation than any other item."^^ 

Despite evidence correlating debate success and high scores for refutation, there is still 
doubt as to whether it is refutation alone that actually accounts for this. In fact, Sanders has 
gone on to suggest that rebuttal may be equally or more important than simple refutation. He 
noted: "Rebuttal is the attempted rebuilding of an argument once it has been attacked. It does 
no good to refute an opponent's argumentation if your own case is in shambles."^ Even Faules 
has suggested that refutation may be inherently dependent upon other factors. "The 
presentation," that is delivery, "of refutation will decide its potency. "^^ He has also noted that 
the whole process of refutation is "dependent upon a student's ability to examine evidence, 
reasoning, and the relationship of evidence and inference. "^^ 

Apparently, refutation is a critical element relating to debate success, but one dependent 
upon other factors as welL Faules, for example, has clearly drawn an interrelationship between 
reasoning, evidence, and refutation. Refutation may well be important, but absent its foundation 
in these other factors it may well be impotent. 

The bulk of contemporary literature tends to endorse the six Form C evaluation factors 

as important, but it fails to distinguish any one as being uniquely important absent the other five. 

Indeed, Professor Burgoon has found that: "Debaters who were rated high on any one 

dimension were consistently rated high on the other five."^^ Vasilius and DeStephen have also 

found a lack of independent criteria for debate evaluation. They have noted: 

' Research indicates that debate evaluation is multidimensional, that some 
evaluative dimensions are more important than others, and that the dimensions are 
not independent, despite "boxes" on a debate ballot indicating evaluative factors.^* 
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Burgoon and Montgomery have gone so far as to suggest that broader, general standards 

actually account for evaluation rather than the traditional six. They reported: 

The collapse of previously discovered dimensions into three in this investigation 
is a significant finding. It implies that when respondents are asked to reveal their 
standards for evaluation rather than to rate actual people, a different judgmental 
structure appears. When evaluating actual people, it seems possible to distinguish 
among composure, sociability, and character attributes. However, when the ideal 
is to be rated, all of these attributes ceem to be intertwined. The logical 
extension of this finding is that judges probably only evaluate debaters along these 
three general lines rather than making six independent judgments, as presumed 
by the old Form C ballots.^^ 

Hence, these general lines may be more important than the specific criteria suggested by current 
debate ballots. 

Many judges have taken the option of simply providing a total score for debate 
performance and ignoring the "boxes" occupied by the six traditional factors. In relation to such 
action, Professor Burgoon has written: 

The failure of judges to discriminate among the six elements implies that either 
(1) they are only making a gross, global evaluation, (2) they are unable to 
translate their true evaluation criteria into marking behavior (which reduces the 
utility of the ballots as feedback to debaters), or (3) other factors are influencing 
their decisions. 

The possibility of "other factors," perhaps nonperformance variables, effecting the 

outcome or evaluation of a debate is most pronounced. This is, of course, generally true in 

regard to speech evaluation as Larry Barker has noted: 

The many uncontrollable variables present in the evaluation situation, coupled 
with different concepts of the ideal speech, compound the problem. 

Evaluations of communication behavior appear to be influenced by a combination 
of environmental, perceptual, and hereditary factors that influence human 
judgement."** 

Such factors could obviously influence a judge-evaluator of a debate round. 
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Debate-specific studies have attempted to measure the effect of nonperformance variables 

on the outcome of debate rounds. Professor Wise has offered one example: 

Although wins over a year*s debating will be approximately equally divided, 
affirmative teams score higher on the average on the six scales than do negatives, 
particularly on "organization" and "delivery." The first affirmative rebuttal 
speech and the first negative constructive speeches are "crucial" speeches in a 
standard format debate.**^ 

The particular variables of "side" and "speaker position," however, do not appear to significanfly 

affect the outcome of debates. Sidney Hill found "that the format variables *side of topic* and 

'speaker position' have no significant effect on the overall outcome of intercollegiate debates as 

measured by the dependent variable index of outcome."'*^ Any effect associated with topic side 

would seem to simply reflect pure chance. Halstead concurred by noting: 

These figures indicate, then, that there may be a slight advantage for one side on 
a specific debate question, but that there seems to be no particular advantage for 
Affirmative per se or Negative per se. Even this advantage may be pure chance, 
and it is so slight an advantage that it is not likely to influence the decision in a 
specific debate."*^ 

Two other nonperformance variables have produced more controversial findings as to 
their effect on intercollegiate debates. Those variables are proximity and gender (of debaters 
and of judges). 

"Physical location alone," Brooks has noted, "exerts a powerful influence on amount of 

interaction.... The powerful, almost mechanical, effect of physical distance on friendship patterns 

is consistently documented. "'^^ Brooks has further explained that: 

Both the conclusions of debaters and the conclusions of scholars studying debate 
judging indicate that debate decisions are based on something other than the 
criteria listed on debate ballots. Hidden criteria, sometimes suggested by 
debaters, are social distance and geographic distance.^^ 
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Brooks further reported that "geographical distance was related to debate decisions in a manner 
not predicted by chance in five of the six tournaments" that he studied/*^ 

Hill has also examined the variable of geographical distance, or proximity. Hill noted: 
"Schools normally do a major portion of their season's debating within their National Debate 
Tournament district, thus potentially fostering 'friendship through propinquity'....;.."^^ Hill felt 
such influence was possibly overstated. He noted: "Because these district lines tend to represent 
natural lines of travel and traditional rivalries, the effects due to simple geographical proximity 
might well be over-ridden by the pressures of district loyalty. "^^ Hill further noted that his 
"model indicated that, within any given N.D.T. district, proximity was a negative influence. 
Perhaps, in this case, proximity led to the growth of rivalries rather than friendships."^^ 

The variable of gender has inspired even greater controversy among forensic scholars. 
For example, Hayes and McAdoo have found gender to effect speaker rankings beyond simple 
chance. They reported: 

The conclusion is that in debates involving at least one mixed team, the rankings 
received by both males and females systematically differ from those expected by 
chance. Under these conditions females receive more "one" and "three" rankings 
but fewer "twos" and "fours." At the same time males differ from chance in that 
they receive more "twos" and "fours" but fewer "ones" and "threes. "^^ 

It has further been suggested that gender can affect total outcome (win/loss), not only individual 
rankings. Rosen, et al found "there is no difference between male and female teams with regard 
to winning, but mixed teams are more likely to win."^^ 

Some authorities feel that the success of male-female teams actually reflects other factors 
at work. Hensley and Strother reported: 

At least two reasons can be advanced for the advantage of the male-female teams. 
First, there may be instances when the respective styles of the male and female 
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tend to complement each other better than if members of the same sex were 
debating as colleagues. Secondly, while in truth, there may be no difference in 
the abilities of the two sexes, coaches may be reluctant to pair a male and a 
female. 



Hensley and Strother further suggest that single gender teams are neither more or less 

successful. The results of their study fails "to give any credence to the superiority of a team 

composed of two males or to the inferiority of a team composed of two females."^ In fact, the 

success of single gender teams seems to reflect chance alone. Hensley and Strother noted: "By 

the laws of chance alone, debating teams can be expected to win 50% of their debates and, 

indeed, teams composed of two males or of two females have records which conform very 

closely to this expectation."^^ 

The gender of those evaluating speech acts may play some part in how those evaluations 

occur. This has been found to be generally true in the field of speech communication. 

According to Barker: "A meaningful relationship was found between instructor's speech ratings 

and the sex of the communicator."^^ In relation to debate. Hill found that "female debaters 

tended to be associated with lower team ratings than did male debaters. Conversely male judges 

tended to give lower team ratings than female judges."" Hill went on to explain the expected 

ratings involved in various situations: 

This model indicates that the members of mixed teams received lower ratings than 
either all-male or all-female teams. Before a male judge, the expected speaker 
rating for the male member of a mixed team was 19.50, as compared to 22.80 for 
a male debater with a male colleague before a male judge. The expected rating 
was 19.12. When debating before a female judge, the female in a mixed team 
had an expected rating of 19.33.^^ 

Hill went even further to suggest that: 

....for any given debate, then these results indicate that all-male teams had a 
greater expectation of winning before a male than before a female judge. Mixed 
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teams and all-female teams, however, had an expected loss trom male judges and 
an expected win from female judges. 

Hence, gender of the judge in relation to gender of the debaters involved may well influence 

evaluations made by those judges. 

The inconsistency and uncertain ty_ surrounding debate speaker evaluation is obvious. 
Many scholars have advocated increasing the amount of empirical and quantitative research 
conducted in the entire field of forensics,^ This would seem to be one area ripe for such 
investigation and analysis. 

Indeed, the need for data specifically relevant to success in debate is most pronounced. 
Burgoon and Montgomery have noted: 

....the controversy over what constitutes superior debating has generated much 
speculation and prescription but very little empirical verification. Debaters and 
judges alike are still uncertain of the universal standards (if any exist) by which 
debaters are evaluated during debate competition.^^ 

Burgoon went on to be more specific in advocating further research: 

....more research using multivariate techniques is needed to obtain a realistic 
assessment of what factors generate success in debate. Efforts should be made 
to combine the traditionally identified factors with such variables as geographic 
biases, sex, reputation of the team's school, source credibility, and refutation 
forms so that relative influence of each can be determined.^^ 

Williams and Webb have stated that "there is little research evidence that lends insight 

into the actual bases for judges' decisions. "^^ This confirms what has been indicated all along: 

there is little knowledge as to what elements actually affect evaluation. The need for such 

information was underscored by Brooks, who reported: 

An integral part of learning is evaluation and feedback. In the educational 
process we assume that evaluation is a rational act involving systematic analysis 
and judgment based on relevant criteria, and that the evaluation should be fed 
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back to the learner so that appropriate understandings and behaviors are positively 
reinforced and erroneous understandings and behaviors are corrected.^ 

The educational necessity for evaluative feedback was confirmed by Professor Burgoon: 

Certainly if students are to learn what elements truly contribute to effective 
ai'gumentation and specifically to successful intercollegiate debate, we must 
identify those factors that are relevant and those that deserve the most emphasis/^ 

Verderber summarized the concept best by stating: "Intercollegiate debate should be an 

educational experience; anything that can be done to improve its value is worth the time and 

effort."^ Hence, if further study were to aid the evaluation and feedback process for debate it 

would be well worth the effort. 

Further research may also aid in the overall process of training debaters. "Training 
procedures," noted Willmington, "varied widely, and it seemed that the type of training a coach 
gave to his debaters depended more upon his whims than upon any consensus as to good training 
procedure. "^'^ Whim would certainly seem to be an insufficient approach to debate and 
argumentation training. Further research into the variables affecting debate success would offer 
a more reasoned alternative to whim alone. 

Hill has concluded that "judges simply don't check the boxes any more."^^ In other 

words, the traditional evaluation technique provided on Form C debate ballots is being 

increasingly ignored. There has even been movement toward abandonment of the "boxes" 

entirely. Whether this is a wise option or not is a question that also warrants additional 

research. The risks associated with an abandonment induced by insufficient research were 

explained by Burgoon: 

Abandoning the Form C~type ballot, however, may mean losing valuable 
information about what factors in reality determine debate success. If, in fact, 
the six components of evidence, organization, reasoning, analysis, refutation, and 
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delivery are critical factors, we need to know three things: how much of the 
success they actually account for, what the relative importance of each is, and 
how independent the judgments are.^^ 

The current trend toward holistic spealcer evaluation may be premature. 

On a very pragmatic basis, the knowledge of what constitutes successful debating may 
be extremely important to the very existence of a debate program. In a period of budget- 
slashing and belt-tightening, few programs that cannot demonstrate their success and worth can 
avoid becoming the victims of such actions. Benson and Friedley note that "obtaining equitable 
funding and staff to coach.... may be intrinsically tied to producing empirical data related to the 
activity's functions and claimed benefits.''*^ Hence, an understanding of what factors actually 
make up the "good person debating well" may be the key to survival of the debate process itself. 

It should be apparent that some form of additional research is warranted. Although the 
trend is toward holistic evaluation, such a movement might not be entirely justified. Independent 
of empirical research, debate scholars can take additional action. For example, it might be an 
interesting personal experiment forjudges confronted with a ballot that offers "boxes" to attempt 
to mark them and to see if that evaluation is consistent with their holistic impression. Coaches 
and judges can also discuss these evaluative variables among themselves and with the debaters 
being evaluated. Such discussion probably will not create a consensus regarding evaluation. 
It might-as this effort has attempted-provide an important heuristic tool for the debate 
community. 
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